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learning environment designed to safely practice and rehearse the daily 


activities related to the school world in children affected with autism. A total 
Keywords: of nine children with autism actively enrolled in the study to learn and test 
their street crossing skills and social attention. Incremental change of 
difficulty levels has been added to the designed environment to generalize 
real-world situations, this includes overlaid distraction audio and increased 
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Pedestrian safety vehicles intensity and speed. In order to enhance the learning experience, the 
Unity real-time feedback is given according to the participant’s behavior, 
Virtual reality additionally, post processing profile is given for analysis purpose, where the 


participant’s behavior can be reviewed by parents and therapist to determine 
whether the participant’s mistakes are in decision making or focusing 
attention. The Wilcoxon signed-rank test for a single sample was used to test 
the change in the skills of participants with autism after using the 
educationally and therapeutically VR technology compared to a baseline. As 
a result, significant effects were found on the behavioral measures indicating 
that the VR-based learning environment is promoting a positive and 
informative learning environment. 
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1. INTRODUCTION 

Autism spectrum disorder (ASD) includes conditions that were previously thought to be separate 
illnesses, these conditions include: autism, Asperger’s syndrome, childhood disintegrative disorder and 
unspecified form of pervasive development disorder [1]. According to the American Psychiatric Association 
(APA) ASD refer to a group of neurodevelopmental disorders that involve a delay in the development of 
many basic skills and functions [2]. This is not just a single disorder but, in fact, a group of closely related 
disorders that have a common core set of symptoms. Social issues are one of the most common symptoms 
throughout all the different types of ASD, as every individual on the autism spectrum has some degree of 
predominant difficulty with social interaction, communication, empathy and flexible behavior. 
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The prevalence of ASD is estimated to affect about 62.2 million globally as of 2015. Although ASD 
begins in early childhood, when symptoms usually are seen by the age of two years, eventually these 
symptoms last throughout an individual’s lifetime. For individual with ASD, the impairments can make life 
very challenging to how conduct crucial activities of daily living. Yet, there is no cure for this disorder, but 
early intensive therapy can significantly improve the lives of many children. The goal of treatment is to 
develop the ability of the child to function by reducing symptoms and supporting learning and development. 
Treatment options contain therapy such as communication and behavioral therapy, family therapy, speech 
therapy and educational therapy. Overall, the type of therapy depends on the child’s need [3]-[5]. 

In recent years, different studies have stated that the virtual reality (VR) intervention can be used 
across different applications in order to improve social skills in children with ASD. The review in [6] has 
carried out an evidence-based systematic review about the effectiveness of VR for children with ASD. The 
review has addressed thirty-one research papers contribute to the improvement of activities in daily life and 
communication for children with ASD, these activities include: social, emotional, daily living, 
communication, attention, physical and phobia. 

The term VR applies to “computer technologies that use interactive software to generate realistic 
images, sounds and other experiences that mimic a real-world context” [7]. In a broader sense, VR is a 
trending term that revolutionizes the present and the future by creating new interactive experience. VR has 
emerged as an innovative new approach in different areas such as healthcare field for rehabilitation and 
treatment [8], [9], education field for learning and teaching [10], [11], also in business field for employee 
training and product development [12]. 

Specifically, VR-based therapy is being used to develop the skills of autistic children for 
independent living. Comparatively, VR intervention offers several benefits compared to traditional 
intervention [13]. First, it can provide safe environment to practice a number of social scenarios in an 
iterative manner [14]. Second, it can provide supportive environment to make social mistakes in challenging 
interaction scenarios in a less-anxiety feelings [15]. Finally, it can provide stimulating and rewarding 
environment to satisfy the individual’s needs [16], [17]. 

In recent few years, since 2015, different solutions have been used to train safe street crossing skills 
through the use of VR among children with autism. In this paper, we develop such a fully immersive 
environment that will generalize to real-world situations, where autistic children become accustomed to 
safely practice and rehearse their daily activities related to the school world. In this environment, the autistic 
children will be actively involved to learn and test their street crossing skills and social attention, and 
real-time feedback is given accordingly to enhance the learning experience. 

This paper is organized as follows. Section 2 presents the background of existing safe street crossing 
training approaches in virtual environment. Section 3 is the development of the VR learning and testing 
environment. Section 4 discusses about the application evaluation and results. Section 5 summarizes how the 
research objectives being achieved, its contribution and future works. 


2. RELATED WORKS 

Lately, some VR-based studies are oriented toward the safe independent living for autistic children, 
such as cross street safely [18], as most of serious injury and death may be caused by traffic accidents, due to 
pedestrian limitations of traffic knowledge, perceptual, and conscious. Therefore, in this section, a number of 
related works, which are based on different technology for planning pedestrian safety, are reviewed in 
general. 

A study [19] was conducted to evaluate several methods which could enhance learning environment. 
These methods are varied from demonstrating educational videos and board game, basically, applying 
classroom activities which are based on street intersections model and a doll to demonstrate the actual reality, 
and providing some real practices in a protected street-environment. The study concluded that the most 
factors that could affect the teaching procedure are reinforcement, prompting, and role-playing. 

Other studies were conducted to evaluate the effect of interactive multimedia programs to educate 
children on how to distinguish the dangerous situations when crossing the street. The main idea of these 
programs is to demonstrate real life situation to the children in a safe way by teaching them some of the 
critical skills that are necessary to cross the street safely. The results of these studies indicated that these 
kinds of programs may increase the efficiency of educating children the proper way to cross the street under 
several circumstances. They also suggested integrating the behavioral-intervention strategies along with 
awareness training to enhance the effectiveness of these kinds of the programs [20], [21]. 

Moreover, the study in [22] was discussed the VR effects of teaching persons with ASD some 
pedestrian skills by providing them a path with signs to guide them on how to cross the street using the 
shortest path. Other studies have discussed the effect of VR on educating persons with ASD these skills by 
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using VR learning facilities. These facilities supposed to educate persons with ASD on how to cross the street 
by demonstrating a protected real-life environment [23]—[25]. The main conclusion of these studies indicated 
that VR may play a significant role in educating persons with ASD the way of how they should cross the 
street safely. 

In a recent study [26], some VR facilities have been applied on the experiment to teach three 
children with ASD how to cross the street safely. The results of the experiment showed that VR plays an 
important role on teaching how to cross the street in efficient way. Additionally, the study suggested that VR 
facilities need to be improved to provide more effective tool for teaching these children the proper way of 
how to safely cross the street, along to other safety skills. 

One more study [27] was also conducted to evaluate the role of VR in how to educate children with 
ages ranging from 5 to 14 years old, on how to safely cross the street. The study applied an experiment on 
these children starting with observing those crossing real streets, then educating those using VR facilities to 
analyze the effectiveness of VR teaching on them. After involving these children in VR cross-street 
education course, they observed them again while they were crossing real streets. They compared the results 
of the tow observation and they concluded that, VR course enhances the abilities for the children to cross the 
street safely with a variety of enhancement related to the children ages. Based on the above-mentioned 
related works in this area, our contribution in this work is developing together a VR-based training and a 
VkR-based testing environment, with a sufficient sample of participants conducted the study of using the 
developed system. Besides, post processing profile of participants will be available for analysis purpose. 


3. RESEARCH METHOD 

We conducted a study of nine children with ASD using the developed system to test our hypothesis. 
The developed system involved multiple phases. The phases including VR-based learning practice, VR-based 
test practice, and finally, VR-based tour practice. 


3.1. Participants 

Nine children with ASD ranging in age from 8 to 11 years (Mean 10.3 years, standard deviation 
1.15) involved in this study. The group included 3 females and 6 males. All children had similar initial levels 
of daily life skills and their parents confirmed that they are totally dependent children and in need of 
supervision. 


3.2. Settings and materials 

We developed the virtual city environment based on unity platform [28], which is one of the most 
widely used technologies that are created for games and VR development purpose in the first place. It can 
help developers create three-dimensional graphics as well as animations that offer visual and functional 
effect. In this paper, the developed virtual city environment was designed with vast array of buildings, streets, 
transportation, people, trees, flowers, and rocks to add variety into scenes and to create more native feel to 
the environment. Picture from this environment is presented in Figure 1. 

The developed VR training and testing environment viewed on a 15-inch LCD monitor. The child 
behavior was tracked using Google Cardboard. Head motion-capture technology was incorporated using 
Google cardboard to capture and project the child’s head and movements into the virtual environment. All 
nine participants were required to use a device like a thick pair of goggles called Google VR headset (Gvr 
headset) to access positional tracking features alongside a pair of headphones to enhance their simulated 
experience of being in another world. In this technology, two pointers were used, one called Google VR 
reticle pointer (Gvr reticle pointer) which is used for the pointer and reticle to follow the user’s gaze (using 
for interaction with the environment), and second one called Google VR base pointer (Gvr base pointer) 
which provides methods called on pointer interaction with in-game objects and user interface (UI), trigger 
events, and “BaseInputModule” class state changes. In this paper, smooth workflow was conducted in 
multiple phases; each phase is discussed in turn. 


3.2.1. VR-based learning practice 

A conceptualization video was produced to depict a rendition of the street beyond the school world. 
This video tends to be primarily educational material to educate the viewer on specific instructions to cross 
the street accompanied by traffic lights as pre-activity demonstration. Traffic lights are positioned at road 
intersections and pedestrian crossings to be integrated into the 3D modelling for simulating flow of traffic 
and pedestrians. Different situations to cross the street were shown either safe or unsafe situation. Also, 
additional distraction audio was laid over the video to serve as arousal level control and barrier of attention. 
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3.2.2. VR-based test practice 

In reflecting upon the learning and training outcome followed in the previous phase, it is clear that 
there is need to bring gaining training ideas to practice in a context beyond the school world. Therefore, in 
this phase, the children will be actively involved in fully immersive VR environment that simulates the world 
of outer school. The child was represented by an avatar being to examine the street crossing, and the 
behaviors that are being recorded are: i) what is the standing point on the sidewalk and distance from the 
crosswalk; ii) whether the participant is looking left and right; iii) how the child responded to the traffic light; 
and iv) how long it takes for the user to respond and cross the street. 

After first round of crossing the street, incremental change of difficulty was then added to determine 
the child behavior to cross one more intersection as a second round. This round included overlaid distraction 
audio, as sounds of pets, lawn mowers and vehicles trumpet. In addition to increase in pedestrians and cars 
intensity, also higher cars speed. These two rounds allowed us to get anecdotal data pertaining to the 
participants’ reactions to the different situations. 


3.2.3. VR-based tour practice 

In this phase, the participant rides the virtual school bus, and the behaviors that are being recorded 
are: i) social interaction, ii) the right seat selection, and iii) using the bus stop signal. As a sequence of 
aforementioned phases as shown in Figures 2 to 4, after the training in the first phase, the participant will be 
simply asked to login to the model in order to record the responses outcome of the remaining two phases. 


rinmi juni 


Figure 1. Screenshot of the virtual environment used Figure 2. Screenshot of the virtual learning 
in the study environment 


Figure 3. Screenshot of the virtual street crossing Figure 4. Screenshot of the virtual tour 
environment 
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It is worth to mention that the user’s access history is archived in the database. So that, it can be 
used as post processing profile for analysis purpose, where the information can be reviewed and analyzed by 
the child, parent, and professionals. Children may also be able to see their own success and weaknesses 
whether it is in decision making or focusing attention. The user’s progress can be followed over time to see 
which skills have improved and which skills still need develop. Parents and therapists can review the 
behavior with the child and point out more appropriate responses and/or behaviors, promoting a positive and 
informative learning environment. 


4. RESULTS 

All nine participants with ASD completed the designated VR-based daily living tasks. The 
performance of all participants is reported in Table 1, in which we compare the baseline score and the post 
intervention score. The Wilcoxon signed-rank test [29] for a single sample was used to test the change in the 
skills of participants with ASD after using the educationally and therapeutically VR technology compared to 
a baseline. The Wilcoxon signed-rank test uses the sum of the signed ranks as the test statistic W. 


W = Ph [sgn(x2,i — x1, i). Ri] (1) 


Table 1. Differences within the participants before and after VR 


Participant Baseline Post intervention Difference Differencel Rank Negative Positive 
number score score (Before-After) signed rank signed rank 
1 16 15 -1 1 1.5 1.5 - 
2 17 20 3 3 3 - 3 
3 7 14 7 7 7.5 - 7.5 
4 8 15 7 7 7.5 - 7.5 
5 13 14 1 1 1.5 - 1.5 
6 12 17 3 5 5.5 - 5.5 
T 5 10 5 5 5.5 - 5.5 
8 7 7 0 0 - - - 
9 10 14 4 4 4 4 


W1=1.5 W2=34.5 


Based on the results which are tabulated in Table 1, we find that W- is 1.5 and W+ is 34.5, and so 
the smaller of these values is the test statistics T. Here we use a (the significance level)=.05 and N=8 (the 
nine subjects excluding the subject number eight where the difference value is zero), hence from the 
Wilcoxon signed-ranks table shown in Table 2, the critical value Tcrit for the T statistic is 3. Since 
T statistic=1.5 is less than Tcrit=3, clearly, this is sufficient evidence to suggest that there is a difference 
between the two measurements scores in terms of daily living tasks improvement. For this test we use the 
following hypotheses: 

- HO: There is no difference between the following baseline and the post intervention. 

- HJ: There is a difference (the median change was non-zero). 

From statistically standpoint, as we have statistically significant evidence at a=0.05, we can reject the null 
hypothesis HO and accept the research hypothesis H1, and so conclude the VR-based training efficacy in 
developing the daily life skills among autism’s children. 


Table 2. Critical values of T for a two-tailed test 
N Two-Tailed Test 
a=0.05 a=0.01 


onno 
v=o }t 


4. CONCLUSION 

This paper presents VR-based learning environment, which provides opportunities for children with 
autism to develop necessary daily living skills in a safe and controlled environment. We effectively 
demonstrated fully immersive VR environment that simulates the world of outer school. In this regard, three 
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phases sequentially were demonstrated; including: VR-based learning practice, VR-based test practice, and 
VR-based tour practice. Future studies will explore the effectiveness of VR technology through a larger 
sample and to children with autism younger than 8 years, also, VR intervention could be extended to more 
general skills, such as conversational and physical skills. 
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